10. Evaluating Performance as Data Scientist

heading

Evaluating Performance as a Data Scientist

ND320 C3 L3 08 Performance Evaluation - Data Scientist Perspective

Video Summary

We have discussed four metrics that you can use to evaluate the performance of your segmentation models. As usual, a great explanation of these can also be found on Wikipedia which I’m linking here if you are looking for additional details:

Note these metrics as they are very handy as you are publishing your model’s validation reports, but also they could be used to construct more elaborate cost functions. We will take a closer look at how these metrics work, but for now, let’s see how clinicians think of performance.

Metric selection

You're building a ML model to segment blood vessels in chest CT scans. Since blood vessels are only a few voxels in diameter, it's possible that the predicted shape might be very similar to ground truth but predicted voxels will not match GT precisely. Which of the following metrics would be best if you wanted to rate this type of prediction similarly to one that labels all the voxels precisely?

SOLUTION: Hausdorff Distance